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1.0  EXECUTIVE  SUMMARY 


The  MetalMapper  (MM)  is  an  advanced  transient  electromagnetic  (TEM)  system  for  application 
towards  the  detection  and  characterization  of  unexploded  ordnance  (UXO).  The  antenna 
configuration  includes  three  orthogonal  transmitter  loops  and  seven  tri-axial  receiver  loops.  The 
system  can  be  deployed  in  mapping  (or  detection  configuration)  wherein  it  acquires  data  along 
profiles  while  the  antenna  platform  is  in  motion.  However,  the  most  important  benefit  of  the 
elaborate  antenna  configuration  is  that  it  permits  us  to  characterize  a  buried  metallic  target  from 
measurements  at  a  single  spatial  point  located  (approximately)  above  the  target.  This  system  is 
being  commercialized  by  Geometries,  Inc.  (San  Jose,  CA). 

In  2008,  the  MM  was  demonstrated  at  the  Standardized  UXO  Technology  Demonstration  site 
located  at  Aberdeen  Proving  Ground  (APG).  In  2009  and  2010,  the  MM  participated  in  live  site 
demonstrations  at  the  former  Camp  San  Luis  Obispo  (SLO),  and  the  former  Camp  Butner.  The 
perfonnance  objectives  of  these  demonstrations  were  to  demonstrate  the  capabilities  of  the  MM 
when  operating  both  in  its  mapping  mode  (target  detection)  and  in  its  static  mode  (cued  ID  target 
identification). 

The  system  has  a  detection  “footprint”  of  approximately  1  m  ,  similar  to  that  of  the  EM61.  In 
detection  mode  surveying,  the  MM  surveyed  along  parallel  profiles  with  a  nominal  offset  of  0.75 
m.  We  acquired  data  at  an  average  survey  speed  of  less  than  0.5  m/s  in  order  to  maximize  data 
quality.  At  those  speeds  our  average  survey  production  was  slightly  more  than  1  acre  per  day. 1 2 
Analysis  of  detection  scores  and  the  MM  data  show  that  at  these  speeds  we  can  detect  targets 
down  to  a  depth  of  1 1  times  the  target  diameter,  as  long  as  the  detection  threshold  is  chosen 
carefully.  Simple  threshold  detection  is  clearly  inadequate  for  instruments  such  as  the  MM,  and 
improved  methods  are  being  developed  in  subsequent  Strategic  Environmental  Research  and 
Development  Program  (SERDP)  projects,  such  as  MR-1772.  Static  mode  survey  productivity  is 
very  much  a  function  of  terrain,  navigation  software,  and  deployment  platfonn.  Over  the  course 
of  our  three  full  demonstrations  (APG,  SLO,  and  Camp  Butner),  the  productivity  of  the  MM  in 
the  static  mode  doubled  from  approximately  200  points  per  day  (pts/day)  to  over  400  pts/day, 
reflecting  improvements  both  in  our  navigation  software  and  in  our  deployment  platfonn. 

Our  performance  improved  from  one  demonstration  to  the  next.  We  attribute  this  improvement 
not  to  improvements  in  overall  data  quality  but  to  improvements  in  our  ability  to  identify  and 
extract  the  important  discrimination  features  from  our  static  target  data  and  in  an  improved 
understanding  of  the  technology  of  decision  theory  and  pattern  recognition.  At  APG,  for 
example,  we  achieved  a  discrimination  score  at  our  operating  point  of  a  Probability  of  Correct 
Classification  {'Pfsc)  of  approximately  90%  at  a  Probability  of  False  Positive  (Pfp)  of 
approximately  10%  where  the  low  probability  of  detection  (Pd)  primarily  represented  deep 
targets  that  were  not  detected  by  the  MM.'  In  contrast,  SLO  discrimination  scores  were  based 


1  Sample  rate  is  typically  10  samples  per  second,  and  the  MM  survey  speed  can  be  easily  doubled  or  tripled  with  a 
corresponding  diminution  of  data  quality  and  spatial  sample  density. 

2  To  preserve  the  ground  truth,  the  Aberdeen  Test  Center  (ATC)  scores  provide  only  a  depth  range.  Most  of  the 
targets  not  detected  at  APG  lay  in  the  depth  range  8x  to  12x  the  object  diameter.  At  SLO,  where  we  had  the 
opportunity  for  a  retrospective  analysis,  such  targets  were  clearly  seen  in  the  data  at  levels  below  the  chosen 
threshold.  We  believe  the  misses  at  APG  were  due  to  a  similar  cause. 
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only  on  detected  targets.  At  SLO,  the  Geometries’  discrimination  score  at  the  operating  point 
was  Pd=98%  with  a  Pfp~5%.  Using  the  same  data,  other  “data  processing  demonstrators” 
generated  similar  results,  thus  showing  that  discrimination  perfonnance  is  not  tied  to  a  particular 
discrimination  methodology.  At  Camp  Butner,  our  discrimination  scores  were  good,  while  those 
of  some  other  demonstrators  were  nearly  perfect.  We  conclude  that  the  data  quality  gathered  by 
the  instrument  is  very  high,  and  sufficient  to  support  further  work  in  processing  of  the  targets  for 
discrimination. 

The  estimated  cost  of  deploying  the  MM  based  on  the  demonstrations  we  have  conducted  is 
high,  two  or  three  times  what  it  would  cost  to  deploy  commercially  available  systems  such  as  the 
EM61.  These  estimated  costs  are  high  because  our  demonstrations  were  conducted  by-and-large 
with  high-cost  personnel,  and  we  conducted  field  operations  to  maximize  data  quality  rather  than 
productivity.  We  project  that  the  cost  per  crew  day  will  drop  substantially  when  the  MM  is 
routinely  deployed  and  the  labor  costs  drop.  As  it  is  now,  the  MM  costs  approximately  $10,000 
per  hectare  (ha)  when  used  in  the  detection  mode  and  $20/target  when  deployed  for  cued 
identification  (ID).  Detection  mode  productivity  is  about  0.5  ha/day;  however,  at  nonnal  survey 
speeds  (~1  m/s),  the  detection  mode  productivity  would  be  1  ha/day.  Cued  ID  mode  survey 
productivity  is  300  targets/day. 

Higher  costs  and  lower  productivity  in  the  detection  mode  can  be  justified  only  when  we  can 
demonstrate  that  through  more  advanced  detection  methods  the  MM  can  eliminate  many 
superfluous  targets  such  as  small  surface  clutter.  So  far,  however,  we  have  not  developed  or 
applied  advanced  physics-based  target  detection  principles  and  therefore  we  cannot  justify  any 
claims  of  superior  target  detection  perfonnance  relative  to  simpler  systems. 

The  demonstrated  advantage  of  deploying  the  MM  lies  with  its  ability  to  discriminate  targets  and 
thus  to  produce  a  prioritized  dig  list.  With  regulatory  acceptance,  the  use  of  a  prioritized  dig  list 
would  reduce  the  number  of  digs  by  50%  or  more  (depending  on  how  conservative  the  dig  policy 
was  set).  This  would  result  in  an  overall  cost  savings  of  30-40%  in  the  cost  of  geophysical 
surveying  plus  digging.  Therefore,  the  high  cost  for  the  deployment  of  the  MM  is  justified  on 
the  basis  that  it  can  substantially  reduce  the  cost  of  digging.  However,  routine  deployment  of  the 
MM  will  require  not  only  broad  regulatory  acceptance  of  this  advanced  electromagnetic  (EM) 
technology  but  also  that  it  or  other  advanced  systems  be  specified  for  use  in  upcoming  UXO 
remediation  projects. 

The  MM  is  now  a  commercially  available  product  from  Geometries.  Geometries  is  able  to 
provide  not  only  the  basic  hardware  but  also  the  software  to  support  the  data  interpretation,  as 
well  as  training  and  field  support. 
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2.0 


INTRODUCTION 


2.1  BACKGROUND 

There  are  millions  of  acres  of  land  located  at  formerly  used  defense  sites  (FUDS)  and  military 
installations  closed  under  the  Base  Realignment  and  Closure  (BRAC)  program.  Much  of  this 
land  has  been  contaminated  with  UXO.  These  lands  must  be  decontaminated  and  restored  before 
they  can  revert  to  civilian  use.  With  present  technology,  the  cost  of  the  decontamination  is 
estimated  to  be  on  the  order  of  10’s  of  billions  of  dollars.  The  high  cost  for  decontaminating 
these  sites  is  driven  by  the  need  to  visually  identify  all  metal  targets  detected  with  current  metal 
detecting  technology.  However,  only  a  few  percent  of  these  targets  are  actually  UXO.  Because 
of  the  potential  danger,  these  digs  must  be  conducted  by  explosive  ordnance  demolition  (EOD) 
technicians  resulting  in  digging  costs  in  excess  of  $  100/target.  At  many  sites,  the  target  density 
can  exceed  100  targets/acre.  Development  of  technology  that  can  reliably  classify  metal  targets 
at  a  cost  that  is  substantially  less  than  the  cost  to  dig  can  therefore  substantially  reduce  the 
overall  cost  for  decontaminating  these  sites  by  reducing  the  number  of  required  digs  [2]. 

The  MM  is  an  advanced  electromagnetic  induction  system  (EMI)  configured  for  the  detection 
and  characterization  of  UXO.  This  technology  represents  a  significant  departure  from  existing 
commercially  available  EMI  technology  (e.g.,  the  Geonics  EM-61).  Although  the  MM  can  be 
used  in  the  survey  or  mapping  mode  for  target  detection,  its  most  important  application  is  for 
cued-ID  target  characterization.  In  the  cued  ID  mode,  data  are  acquired  at  one  or  more  locations 
in  close  proximity  to  the  target.  Precision  static  measurements  at  these  locations  pennit  the 
calculation  of  the  target  characteristics.  For  isolated  targets,  a  measurement  at  a  single  field 
point  suffices.  These  target  characteristics  may  be  used  to  generate  a  prioritized  dig  list  that  can 
be  used  to  identify  targets  that  can  with  high  confidence  be  left  in  the  ground  because  they  are 
not  ordnance. 

2.2  OBJECTIVE  OF  THE  DEMONSTRATIONS 

The  primary  objective  of  the  demonstrations  perfonned  in  this  project  was  to  gain  acceptance  of 
the  MM  system  by  the  UXO  community,  including  the  regulators.  To  achieve  this  objective,  we 
have  shown  by  these  blind  demonstrations  that  the  MM  can  indeed  be  used  to  reliably  prioritize 
target  lists  for  digging  and  significantly  reduce  the  number  of  false  positive  (i.e.,  non-ordnance) 
items  that  need  to  be  excavated. 

2.3  REGULATORY  DRIVERS 

While  there  are  no  regulatory  barriers  to  applying  the  technology  embodied  in  advanced  EMI 
systems  such  as  the  MM,  the  real  problem  is  to  convince  regulators  and  other  stakeholders  of  the 
reliability  of  the  discrimination  information  that  the  MM  can  provide.  The  Environmental 
Security  Technology  Certification  Program  (ESTCP)  program  office  (PO)  is  addressing  this 
issue  by  conducting  a  series  of  blind  live-site  tests  in  which  promising  new  technologies  such  as 
the  MM  are  invited  to  participate.  The  demonstrations  at  Camp  SLO  and  Camp  Butner  were  two 
of  these  tests.  To  gain  regulator  acceptance,  the  PO  has  involved  regulators  in  the  site  selection 
and  other  aspects  of  the  test  design. 
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3.0  TECHNOLOGY 


The  MM  was  employed  in  full  demonstrations  at  the  Standardized  UXO  Technology 
Demonstration  site  at  APG,  at  the  fonner  Camp  SLO,  and  most  recently  at  the  former  Camp 
Butner.  A  description  is  given  in  this  section. 

3.1  TECHNOLOGY  DESCRIPTION 

Geometries  is  commercializing  an  advanced  EMI  system  for  UXO  detection  and 
characterization.  Called  the  MM,  the  new  system  draws  elements  of  its  design  from  advanced 
systems  developed  by  G&G  Sciences,  Inc.  (supported  by  Naval  Sea  Systems  Command 
[NAVSEA],  SERDP,  and  ESTCP)  and  by  the  Lawrence  Berkeley  National  Laboratory  (LBNL) 
with  support  from  SERDP  and  ESTCP. 

3.1.1  MM  Hardware 

The  MM  system  is  unique  and  innovative  in  several  respects: 

•  Multiple  Transmitter  Loops:  The  MM  antenna  platform  includes  3  mutually 
orthogonal  transmitter  loops  (see  photo  in  Figure  1). 

•  3-Axis  Sensor  Array:  The  MM  antenna  platform  includes  an  array  of  7  3 -axis 
receiver  antennas,  yielding  21  measurements  of  the  secondary  magnetic  field. 

•  Electronically  Switched  TEM  Transmitter  Loop  Driver:  Under  control  of  the 
data  acquisition  (DAQ)  computer,  the  output  of  the  transmitter  can  be  directed  to 
any  single  loop  or  automatically  multiplexed  between  loops.  Typically  the  loops 
are  driven  with  a  classical  bipolar  pulse-type  TEM  waveform  (i.e.,  alternating 
pulse  polarity  with  a  50%  duty-cycle). 

The  photo  of  the  MM  shown  in  Figure  1  was  taken  during  the  demonstration  that  we  conducted 
at  the  standardized  UXO  test  site  located  at  APG.  During  that  demonstration,  we  towed  the  MM 
with  a  garden  tractor  in  order  to  acquire  dynamic  data  at  a  unifonn  survey  speed.  The  system 
may  be  deployed  on  skids  to  minimize  platform  movement  over  undulating  terrain.  The  antenna 
platform,  including  skids  and  optional  wheels,  weighs  approximately  160  lb  (72  kg)  and  can  be 
moved  by  hand  when  necessary.  At  APG,  cued  ID  (static)  data  were  also  acquired  with  the 
antenna  platfonn  deployed  as  a  towed  system.  At  SLO  we  mounted  the  MM  in  a  skid  connected 
to  the  front-end  loader  of  a  compact  Kabota  tractor.  This  deployment  method  improved  our 
productivity  for  static  data  acquisition  since  it  was  easier  to  precisely  locate  the  MM  antenna 
over  a  static  target  site. 

The  DAQ  is  built  around  a  commercially  available  product  from  National  Instruments  (NI).  The 
DAQ,  EM  transmitter,  and  batteries  for  the  entire  system  are  packaged  in  an  aluminum  case  that 
weighs  approximately  43  lb  when  all  four  batteries  are  installed.  For  surveying,  the  aluminum 
case  can  be  mounted  on  a  pack  frame,  on  a  separate  cart  such  as  a  hand  truck,  or  on  a  vehicle 
such  as  an  all-terrain  vehicle  (ATV)  or  lawn  tractor  (see  Figure  1). 
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Figure  1.  Photos  showing  the  MM  antenna  array  together  with  the  electronics  package  and 

control  console  (inset  above). 

The  top  photo  was  taken  at  a  demonstration  conducted  at  APG.  The  bottom  photo  was  taken  at 

SLO. 


6 


The  instrumentation  package  includes  two  external  modules  that  provide  real-time  kinematic 
(RTK)  global  positioning  system  (GPS)  and  platfonn  attitude  (i.e.,  magnetic  heading,  pitch,  and 
roll).  The  NI  data  acquisition  system  is  a  full-featured  personal  computer  (PC)  running 
Windows  XP.  It  contains  disk  storage,  serial  and  universal  serial  bus  (USB)  input/output  ports, 
and  more.  It  also  contains  a  wireless  link  that  allows  the  operator  console  to  be  remote  from  the 
DAQ. 

3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  principal  advantages  of  the  MM  system  over  existing  commercially  available  EMI  system 
are  two-fold: 

1.  Multitransmitter  Target  Illumination:  Because  the  MM  has  three  orthogonal 
transmitter  loops,  it  is  able  to  illuminate  or  stimulate  a  target  with  primary  fields 
from  three  independent  directions  from  a  single  spatial  field  point.  In  contrast, 
single  transmitter  coil  systems  require  the  transmitter  (and  receivers)  to  be 
relocated  to  other  field  points  so  that  the  primary  field  will  stimulate  the  target 
from  a  different  direction.  The  concept  is  illustrated  in  Figure  2. 

2.  Multiple  Multi-Axis  Receivers:  The  MM  employs  multiple  receivers.  The  MM 
receiver  array  consists  of  seven  tri-axial  receiver  cubes  measuring  10  cm  on  a 
side.  With  this  array,  the  MM  is  able  to  sample  the  transient  secondary  vector 
field  at  seven  unique  spatial  points.  The  positions  of  the  receiver  array  are 
provided  in  Figure  4.  The  receivers  are  positioned  so  that  they  traverse  profiles 
13  cm  apart  along  a  survey  line. 


a)  Single  Tx  coil  illumination 


b)  Multi-axis  Tx  coil  illumination 

Figure  2.  Figure  contrasting  the  difference  between  single  coil  and  multi-axis  coil 

transmitter  illumination)!]. 
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4.0  PERFORMANCE  OBJECTIVES 


We  have  conducted  three  full  demonstrations  (APG,  SLO,  and  Camp  Butner).  We  highlight  here 
our  results  at  SLO.  Our  objectives  were  as  follows: 

Table  1.  Global  SLO  classification  study  performance  objectives. 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Detection  of  all 
munitions  of  interest 

Percent  detected  of  seeded 
items 

•  Location  of  seeded  items 

•  Anomaly  list 

At  least  98%  of  seeded 
items  detected 

Maximize  correct 
classification  of 
munitions 

Number  of  targets  of 
interest  (TOI)  retained. 

•  Prioritized  anomaly  lists 

•  Scoring  reports  from  the 
Institute  for  Defense 
Analyses  (IDA) 

Approach  correctly 
classifies  all  TOI 

Maximize  correct 
classification  of  non¬ 
munitions 

Number  of  false  alarms 
eliminated 

•  Prioritized  anomaly  lists 

•  Scoring  reports  from 

IDA 

Reduction  of  false  alarms 
by  >  30%  while  retaining 
all  TOI 

Minimize  number  of 
anomalies  that  cannot  be 
analyzed 

Number  of  anomalies  that 
must  be  classified  as 
“unable  to  analyze” 

•  Demonstrator  target 
parameters 

Reliable  target  parameters 
can  be  estimated  for  > 

90%  of  anomalies  on  each 
sensor’s  detection  list 

Dynamic  Mode  Survey 
Productivity 

Area  surveyed  per  day 

•  Field  production  log 

>1.2  acre/day  @  0.75  m 
Lane  spacing 

Static -Mode  Survey 
Productivity 

Static  pts/day 

•  Daily  production  logs 

Pts  >30/hr  or 

Pts  >200/day 

4.1  OBJECTIVE:  DETECTION  OF  ALL  ITEMS  OF  INTEREST 

Quality  data  should  lead  to  a  high  probability  of  detecting  the  munitions  of  interest  at  the  site. 
The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the  specified 
anomaly  selection  threshold.  The  objective  will  be  considered  to  be  met  if  at  least  98%  of  the 
seeded  items  are  detected. 

4.2  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  MUNITIONS 

By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  we  expect  to  be  able  to  classify  the  targets  with  high  efficiency.  The 
metric  for  this  objective  is  the  number  of  items  on  the  master  anomaly  list  that  can  be  correctly 
classified  as  munitions  by  each  classification  approach.  The  objective  will  be  considered  to  be 
met  if  all  the  items  of  interest  are  correctly  labeled  as  munitions  on  the  prioritized  anomaly  list. 

4.3  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON¬ 
MUNITIONS 

By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  we  expect  to  be  able  to  classify  the  targets  with  high  efficiency.  This 
reduces  the  number  of  false  alarms.  The  metric  for  this  objective  is  the  number  of  items  of 
interest  on  the  master  dig  list  that  can  be  correctly  classified  as  non-munitions  by  each 
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classification  approach.  The  objective  will  be  considered  to  be  met  if  more  than  30%  of  the  non¬ 
munitions  items  can  be  correctly  labeled  as  non-munitions  while  retaining  all  the  TOI  on  the  dig 
list. 

4.4  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce  the  effectiveness  of 
the  classification  process.  The  number  of  anomalies  for  which  reliable  parameters  cannot  be 
estimated  is  the  metric  for  this  objective.  The  objective  will  be  considered  to  be  met  if  reliable 
parameters  can  be  estimated  for  >  90%  of  the  anomalies  on  each  sensor  anomaly  list. 

4.5  OBJECTIVE:  HIGH  DYNAMIC-MODE  SURVEY  PRODUCTIVITY 

Operating  in  a  vehicle-towed  configuration,  we  surveyed  the  entire  test  area  with  the  MM 
operating  in  its  dynamic  acquisition  mode  using  a  lane  spacing  of  0.75  m  and  a  survey  speed  of 
0.5  m/s  (30  m/minute).  Our  objective  was  to  complete  the  dynamic  survey  at  a  rate  of  1  acre  per 
day  or  more. 

Since  the  MM  antenna  array  contains  seven  tri-axial  receiver  cubes  positioned  so  that  each 
receiver  cube  tracks  independent  parallel  profiles,  the  data  from  a  single  profile  acquired  with  the 
MM  can  be  split  into  seven  individual  profiles  for  the  purpose  of  generating  a  very  high 
resolution  detection  map  (13  cm  line  spacing),  even  at  a  relatively  wide  0.75  m  line  spacing  of 
the  vehicle  track. 

The  metric  for  dynamic-mode  survey  productivity  will  be  the  average  area  covered  per  day. 

4.6  OBJECTIVE:  HIGH  STATIC-MODE  SURVEY  PRODUCTIVITY 

When  operating  in  the  cued  ID  or  static  survey  mode,  our  objective  is  to  maximize  the  number  of 
sites  visited  per  day  consistent  with  maintaining  data  quality.  Data  quality  is  primarily  a  function 
of  two  variables  in  the  field.  First  it  is  necessary  to  position  the  center  of  the  antenna  platform  as 
nearly  as  possible  to  the  desired  target  point.  Second,  data  quality  depends  on  the  stacking 
time — longer  stacking  time  means  better  data.  Improving  productivity  therefore  means 
decreasing  both  the  time  for  target  reacquisition  and  minimizing  the  stacking  time. 

For  static-mode  surveys,  the  metric  will  be  targets/hr  . 

Our  criterion  for  success  for  productivity  in  the  static  survey  mode  is  30  targets/hr.  We  achieved 
that  level  of  production  both  at  Yuma  Proving  Ground  (YPG)  and  at  APG.  With  the 
modifications  we  plan  in  our  procedures  and  navigation  software  as  well  as  in  the  way  we  mount 
the  antenna  platfonn  to  survey  vehicle,  we  hope  to  improve  on  the  productivity  we  achieved  in 
our  other  demonstrations. 


3  We  are  adopting  an  hourly  rate  rather  than  a  daily  rate  here  because  field  days  can  vary  significantly  because  of 
site  access. 
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5.0  SITE  DESCRIPTION 


Over  a  one-year  period,  we  have  conducted  demonstrations  at  the  Standardized  UXO 
Technology  Demonstration  Sites  located  at  YPG,  APG,  Fonner  Camp  SLO  in  California,  and 
most  recently  at  Camp  Butner  in  North  Carolina.  This  Cost  and  Performance  Report  focuses 
mostly  on  the  results  of  YPG,  APG,  and  SLO,  since  the  Camp  Butner  analysis  was  ongoing  at 
the  time  this  was  written. 

5.1  SITE  SELECTION 

The  fonner  Camp  SLO,  located  near  San  Luis  Obispo,  CA,  was  carefully  selected  by  the  ESTCP 
PO  as  the  demonstration  site  for  classification  study  to  be  conducted  in  2009.  The  MM  was  one 
of  three  advanced  EM  systems  to  participate  in  that  study.  The  12-acre  study  area  is  a  fonner 
mortar  and  2.36-inch  bazooka  rocket  range.  The  site  contains  four  munitions  types  of  primary 
interest:  2.36-inch  rockets,  60  mm  mortars,  81  mm  mortars,  and  4.2-inch  mortars.  In  contrast 
with  the  standardized  UXO  test  areas  at  YPG  and  APG,  the  SLO  demonstration  area  is  situated 
on  a  hill  and  as  such  presents  a  significant  challenge  for  survey  in  both  detection  or  survey  mode 
acquisition  and  in  cued  ID  or  static-mode  acquisition. 

5.2  SITE  HISTORY 

Camp  SLO  was  established  in  1928  as  a  facility  for  the  California  National  Guard.  It  was  used 
extensively  during  World  War  II  as  a  training  facility  and  again  during  the  Korean  conflict.  The 
facility  was  inactive  after  the  Korean  conflict.  At  the  present  time,  approximately  half  of  the 
original  land  has  been  transferred  to  Cal  Poly  State  University  and  Cuesta  College,  and  the  other 
half  has  been  retained  by  the  California  National  Guard. 

5.3  SITE  GEOLOGY 

In  contrast  to  the  standardized  test  sites,  topographic  relief  over  the  12-acre  SLO  site  is  in  excess 
of  200  ft  with  slopes  sometimes  exceeding  15%.  Soil  textures  and  thicknesses  range  widely. 
Thick  sandy  alluvium  is  found  adjacent  to  washes  and  drainage  channels.  Thin  to  thick 
mountain  terrace  soils  typically  overlay  crystalline  metamorphic  bedrock  and  are  usually 
comprised  of  sandy  to  silty  loam  with  a  thin  veneer  of  silty  clay  near  the  surface.  Moisture 
levels  vary  seasonally.  At  the  time  of  our  demonstration  (May- June  2009),  the  soils  were  very 
dry. 


5.4  MUNITIONS  CONTAMINATION 

The  SLO  site  is  a  live  site.  A  preliminary  dig  of  two  small  50  ft  x  50  ft  grids  in  the  12-acre  area 
of  interest  revealed  4  types  of  munitions:  2.36-inch  rockets,  60  mm  mortars,  81  mm  mortars,  and 
4.2-inch  mortars. 
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6.0  TEST  DESIGN 


6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Each  of  the  field  demonstrations  was  conducted  in  three  phases: 

1 .  Dynamic-mode  survey  for  target  detection 

2.  Static-mode  survey  for  cued-ID  discrimination 

3.  Post-acquisition  analysis. 

The  Gantt  chart  in  Figure  3  provides  a  timeline  for  the  demonstration  and  also  serves  as  an 
outline  for  the  various  activities  involved. 


C)  Activities  -  MetalMapper  SLO  Demonstration  (May-Jun  2009) 


Figure  3.  Gantt  charts  providing  timelines  and  activity  breakdowns  for  MM 
demonstration  covered  in  this  report. 

6.1.1  Target  Detection  (Mapping  Survey) 

In  the  first  phase  of  each  field  demonstration,  the  MM  was  operated  in  its  dynamic  (mapping) 
mode  in  order  to  generate  a  detection  map.  In  its  dynamic  mode,  only  the  horizontal  (Z) 
transmitter  loop  is  energized.  Data  are  acquired  along  a  series  of  parallel  profiles  with  0.75  m 
(nominal)  line  separation.  These  data  are  compiled  into  a  detection  map  wherein  the  mapped 
parameter  indicates  a  peak  centered  directly  over  any  metallic  target  within  range.  Peaks  were 
chosen  (i.e.,  “detected”)  either  manually  using  the  computer  mouse  or  automatically  using  an 
automatic  peak  detector  that  operates  on  a  data  grid.  In  all  cases  where  an  automatic  peak 
detector  is  employed,  it  was  necessary  to  edit  the  resulting  peaks  in  order  to  remove  or  “merge” 
multiple  target  picks  on  the  same  anomaly.  The  resulting  edited  target  list  was  used  during  the 
next  (cued  ID)  phase  of  the  demonstration. 
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6.1.2  Cued  ID  (Static  Survey) 


The  target  list  generated  from  the  mapping  survey  is  “cued”  (sequenced)  so  that  each  target  can 
be  efficiently  reacquired.  Reacquisition  requires  that  the  antenna  platfonn  be  nominally  centered 
over  the  anomaly  of  interest.  Once  a  target  is  reacquired,  the  MM  is  activated  to  acquire  a  single 
static  data  set.  For  static  data  acquisition,  each  of  the  three  transmitter  loops  is  energized  while 
data  transients  are  measured  from  the  seven  triaxial  cube  receivers  (2 1  transients).  Because  the 
antenna  platfonn  is  motionless,  each  receiver  transient  can  be  repetitively  stacked  over  many 
cycles  of  the  transmitter,  thus  significantly  reducing  incoherent  random  noise.  A  complete  data 
set  that  includes  the  results  from  energizing  each  of  the  three  transmitter  loops  takes  25-50 
seconds,  depending  on  the  stacking  parameter. 

6.1.3  Post- Acquisition  Analysis 

Preliminary  data  processing  for  the  purpose  of  generating  detection  maps  and  for  quality 
assurance/quality  control  (QA/QC)  purposes  is  carried  out  throughout  the  course  of  the  field 
demonstration.  However,  most  of  the  data  processing  required  to  assemble  a  target  list  for 
scoring  occurs  after  the  field  activities  have  been  completed  (post-acquisition).  Post-acquisition 
processing  includes  the  following  six  steps: 

1.  Target  Association:  Static  data  points  are  associated  with  a  particular  target  site 
on  the  basis  of  proximity. 

2.  Feature  Selection:  Statistical  analyses  including  scatter  plots,  principal 
component  analysis,  and  trial  and  error  are  perfonned  to  help  select  a  set  of 
features  (feature  vector)  associated  with  the  targets  that  we  think  best  identifies 
the  TOI  from  clutter. 

3.  Library  Matching:  Generate  a  library  of  “type”  polarization  curves  representing 
the  munitions  types  known  to  exist  in  the  area  of  interest.  Match  the  polarization 
curves  extracted  from  each  of  the  unknown  targets  with  those  in  the  type  library. 
Select  the  target  type  with  the  best  matching  score. 

4.  Neural  Network  Training:  Train  a  neural  network  using  feature  vectors 
corresponding  to  targets  for  which  ground  truth  is  available. 

5.  Neural  Network:  Apply  feature  vectors  from  each  target  in  the  test  set  (i.e.,  all 
unclassified  targets)  to  the  neural  network  trained  in  Step  4. 

6.  Target  List  Assembly:  Finally,  the  results  from  both  library  matching  and  neural 
network  analysis  are  merged  to  fonn  a  single  classification  score. 

6.2  SITE  PREPARATION 

The  standardized  UXO  sites  have  been  seeded  with  inert  munitions  according  to  procedures 
reported  in  a  handbook  assembled  as  part  of  the  Standardized  UXO  Demonstration  Program  [3]. 
Unlike  the  standardized  test  sites  at  APG  and  YPG,  Camp  SLO  is  a  live  site.  Preliminary 
investigations  included  a  magnetometer  transect  survey  and  an  EMI  survey  over  a  larger  area  to 
assist  in  selecting  a  smaller  site  suitable  for  the  discrimination  study.  In  addition,  two  small  50  ft 
x  50  ft  grids  were  dug  in  order  to  identify  the  munition  types  that  would  likely  be  encountered 
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during  the  SLO  study.  Finally,  additional  munitions  of  the  type  expected  at  SLO  were  seeded  in 
order  to  increase  the  ratio  of  TOI  to  clutter  items. 

6.3  SYSTEM  SPECIFICATION 

6.3.1  Antenna  Platform 

The  MM’s  three  transmitting  loops  are  positioned  as  follows: 

•  Z  transmitter  (coil  axis  vertical):  lmxlm,  center  is  ~2 1  cm  above  ground  level. 
The  center  of  the  Z  loop  is  taken  to  be  the  local  origin  of  coordinates  for  the  cart. 

•  Y  transmitter  (coil  axis  horizontal  in  direction  of  travel):  lmxlm,  centered 
0.56  m  above  the  origin. 

•  X  transmitter  (coil  axis  horizontal,  clockwise  from  Y):  0.98  m  x  0.98  m,  centered 
0.56  m  above  the  origin. 

The  MM’s  seven  receivers  are  positioned  as  shown  in  Figure  4.  Note  that  the  seven  receivers 
traverse  profiles  that  are  13  cm  apart  in  the  cross-track  (x)  dimension. 
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Figure  4.  MM  receiver  locations. 

All  seven  receivers  are  in  the  plane  of  the  horizontal  transmitter  (Z)  loop  (above  left).  Above 
right  shows  an  MM  acquisition  block  illustrating  a  bipolar  transmitter  wavefonn  with  50%  duty 
cycle  and  a  repeat  factor  (N)  of  3.  Block  periods  are  operator  selectable  from  33.333  ms  to  0.9  s 

in  multiplicative  steps  of  3. 

6.3.2  Data  Acquisition:  Dynamic  Data 

To  collect  dynamic  data,  the  MM  is  operated  in  continuous  mode,  and  all  data  from  a  single 
survey  line  are  stored  in  a  single  file. 

6.3.3  Post-Acquisition  Data  Processing:  Dynamic  Data 

Data  are  stored  as  binary  fonnatted  files.  The  processing  software  includes  the  capability  to 
export  the  data  to  a  Geosoft  Oasis  Montaj  (OM)  database  and/or  to  text  files.  For  dynamic 
surveys,  OM  is  used  for  display,  QC,  map  compilation,  etc. 
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As  we  indicated  in  Section  4  of  this  report,  we  follow  a  two-step  approach  to  target 
characterization.  The  first  step  is  target  detection  and  for  that  we  conduct  a  dynamic  survey. 
Using  a  custom  Geosoft  executable  (GX),  the  data  from  the  dynamic  survey  are  imported  into  a 
Geosoft  database  (GDB).  Other  custom  GXs  have  been  developed  to  process  the  transient  data 
to  a  point  where  they  can  be  further  processed  and  displayed  using  standard  capabilities  within 
the  Geosoft  Montaj  software  package.  The  end  products  of  the  post-acquisition  processing  of  the 
dynamic  data  are  target  detection  maps  and  profiles  and  a  list  of  detected  targets  along  with  their 
respective  X-Y  coordinates.  In  Figure  5,  we  show  the  detection  map  that  we  generated  from 
dynamic  data  acquired  over  the  Blind  Grid  at  APG. 


Figure  5.  Target  detection  map  -  Blind  Grid. 

MM  demonstration  at  APG  (September  2008). 

The  “+”  symbol  denotes  the  center  of  a  2  m  x  2  m  cell. 

6.3.4  Post-Acquisition  Processing:  Static  Data 

Static  data  points  are  acquired  with  the  MM  platform  located  over  each  of  the  targets  identified 
using  the  dynamic  data  as  described  above.  These  data  files  are  processed  with  our  physics- 
based  modeling  program  MM/RbstMultiPrince  (MM/RMP)  (inversion  program)  to  extract 
target-specific  parameters. 

6.4  CALIBRATION  ACTIVITIES 

Calibration  checks  of  the  MM  system  were  perfonned  at  least  two  times  per  day  as  part  of  our 
routine  QA/QC  procedure.  At  SLO,  the  dynamic  calibration  was  perfonned  over  the  Calibration 
Strip.  A  check  on  the  static  measurement  was  perfonned  using  a  steel  sphere  located  in  the 
calibration  pit.  The  calibration  procedure  is  enumerated  below: 

1.  Acquire  a  static  background  data  set  over  a  designated  “background”  point 
(typically  a  designated  point  at  the  beginning/end  of  the  Calibration  Strip. 

2.  Acquire  dynamic  data  over  the  calibration  strip  in  reciprocal  directions  using  our 
standard  dynamic  survey  acquisition  parameters 
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3.  Acquire  a  static  data  set  with  the  antenna  platform  centered  over  a  steel  sphere 
located  in  the  Test  Pit. 

The  calibration  procedure  provides  all  the  data  required  to  perfonn  a  variety  of  QA/QC  checks 
that  document  that  the  instrument  is  functioning  correctly.  Moreover,  the  resulting  data  was  also 
used  to  establish  the  long-tenn  stability  of  the  instrument  response. 

6.5  DATA  COLLECTION 

We  surveyed  the  entire  11.8-acre  study  area  at  SLO  with  the  MM  operating  in  detection  mode. 
Using  maps  from  these  data,  we  selected  2178  targets  for  cued  ID  measurements.  We  also 
repeated  cued  ID  measurements  over  314  of  these  targets  based  on  inversion  results  that 
indicated  that  the  horizontal  offset  of  these  targets  was  greater  than  0.4  m.  We  submitted  a 
detection  target  list  containing  a  total  of  1638  target  picks.  We  were  able  to  associate  at  least  one 
MM  static  file  with  1617  picks. 

6.6  VALIDATION 

Camp  SLO  is  a  live  site.  Detected  items  were  dug  by  Parsons,  Inc.  under  contract  with  ESTCP. 
Parsons,  Inc.  utilized  standard  procedures  for  digging  UXO  live  sites.  In  addition,  the  position, 
depth,  approximate  attitude,  and  a  photograph  of  the  resulting  target  found  were  recorded.  The 
results  of  the  first  five  30  m  x  30  m  survey  blocks  were  provided  to  all  data  processing 
demonstrators  for  use  as  training  data.  The  remainder  of  the  dig  results  were  held  in  confidence 
by  the  program  officer  and  IDA  and  used  for  the  purpose  of  scoring  discrimination  target  lists 
prepared  by  the  demonstrators  according  to  protocols  described  in  a  scoring  memorandum 
prepared  by  the  IDA  and  distributed  to  the  demonstrators  [4], 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 


7.1  PREPROCESSING 

7.1.1  Dynamic  Survey  Data 

The  dynamic  survey  data  consists  of  parallel  profiles  acquired  with  0.75  m  offsets.  Each  profile 
is  recorded  to  a  separate  binary  data  file.  From  these  data,  we  compiled  detection  parameter 
maps  of  the  surveyed  area  or  sub-area.  The  detection  map  is  based  on  the  magnitude  of  the 
secondary  fields  measured  at  each  of  the  seven  tri-axial  receiver  sensors.  The  following 
processing  steps,  accomplished  using  Geosoft  OM,  are  required: 

1.  MM  data  are  recorded  as  binary  files.  These  data  are  imported  directly  into  an 
OM  database  where  simple  editing  is  perfonned  (editing  line  numbers, 
deselecting  duplicate  lines,  trimming  and  deleting  bad  data  or  stops,  etc).  All 
other  steps  are  accomplished  from  within  OM  using  its  standard  editing  and 
processing  capabilities  supplemented  where  necessary  with  custom  GXs,  Geosoft 
Scripts  (GSs),  and  Geosoft  mathematical  expression  (EXP)  files. 

2.  Convert  latitude  and  longitude  to  Universal  Transverse  Mercator  (UTM) 
coordinates,  and  make  a  true  heading  channel  from  the  UTM  coordinates.  Edit 
and  filter  pitch  and  roll  channels.  Compute  corrected  UTM  coordinates  according 
to  heading,  pitch,  and  roll. 

3.  Compute  detector  gate  values  for  each  of  the  21  receiver  channels.  The  detector 
gate  value  is  the  value  of  the  recorded  transient  integrated  optionally  with  an 
exponential  weighting  function  over  a  selected  time  interval. 

4.  Compute  transmitted  current  by  gating  (windowing)  the  transmitted  current 
transient. 

5.  Select  and  remove  background  (leveling). 

6.  Generate  vector  magnitude  channels  for  each  of  seven  tri-axial  receiver  cubes 

7.  Split  each  profile  into  seven  separate  profiles,  corrected  for  heading  and  offset 
distance  from  the  platform  measure  point  (generates  seven  parallel  profiles  with 
13  cm  offsets). 

8.  Grid  the  resulting  amplitude  data,  applying  grid  smoothing  filters  if  necessary. 

Steps  1-5  in  the  above  numeration  constitute  preprocessing  steps  that  result  in  data  that  can  be 
exported  for  delivery  to  the  PO  and  analysis  demonstrators. 

7.1.2  Static  Data 

As  we  stated  previously,  raw  static  data  files  are  input  directly  into  MM/RMP  together  with  an 
appropriate  background  file  and  a  file  containing  root  mean  square  (RMS)  noise  estimates  for 
each  channel  and  time  gate.  What  we  have  heretofore  tenned  “preprocessing”  (e.g.,  location 
correction,  background  removal,  etc.)  is  performed  within  MM/RMP  during  the  parameter 
extraction  process.  Preprocessed  data  were  required  only  during  the  SLO  demonstration. 
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During  that  demonstration,  there  were  several  organizations  that  needed  these  data  to 
independently  prepare  discrimination  target  lists.  We  modified  MM/RMP  so  that  it  can  output 
preprocessed  static  data  files  in  comma  separated  values  (CSV)  format  as  specified  by  the  PO. 
The  data  files  were  delivered  to  the  PO  and  subsequently  distributed  to  other  demonstrators. 

7.2  TARGET  SELECTION  FOR  DETECTION 


Guidance  from  the  ESTCP  demonstration  plan  for  the  SLO  Classification  Study  required  that  all 
targets  be  picked  on  the  basis  of  anomaly  threshold.  The  applicable  detection  threshold  was  to 
be  determined  by  studying  the  predicted  and  measured  response  for  the  four  principal  TOIs  at 
SLO,  which  are  the  60  mm/81  mm/4.2-inch  mortars  plus  the  2.36-inch  (bazooka)  rocket.  Our 
detection  threshold  was  based  on  the  predicted  response  of  the  smallest  of  the  four  TOIs  when 
placed  in  its  least  favorable  orientation  (horizontal)  when  buried  at  a  depth  of  interest  of  45  cm 
below  ground.  We  generally  followed  the  methods  and  rationale  developed  by  Nelson,  et.  al.  [5] 
in  determining  the  threshold  to  apply  for  target  detection  at  SLO.  Lirst,  we  computed  average 
polarizability  transients  for  the  four  TOIs.  We  computed  these  averages  using  static  data  sets  that 
we  measured  with  the  MM  over  the  four  TOIs.  We  were  able  to  use  data  from  YPG  and  APG 
for  the  60  mm  and  81  mm  mortars.  We  used  approximately  15  data  sets  for  each  target  type 
representing  a  range  of  depths  and  orientations  with  respect  to  the  antenna  platform.  The  results 
are  shown  in  Figure  6.  The  smallest  detection  threshold  was  about  3  nanoteslas  (nT)/s  for  the 
horizontal  2.36-inch  rocket  at  a  depth  of  0.66  m  below  the  platfonn  (0.45  cm  below  ground 
level). 
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4.2— in  Mortar  —  SLO  Static  &  Test  Strip 


81mm  Mortar  —  SLO  Static  &  Test  Strip 
too 
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Figure  6.  The  average  principal  polarizability  curves  (red,  green,  blue)  for  the  four 

principal  TOIs  at  Camp  SLO. 

These  curves  were  derived  from  approximately  1 5  independent  static  MM  measurements 
acquired  in  the  free  air  (SLO  Test  Pit),  and  in  the  ground  (SLO  Test  Strip).  The  average  curves 
were  used  in  estimating  the  target  detection  threshold  to  be  used  at  SLO. 


20 


7.3  PARAMETER  ESTIMATES 


The  conceptual  processing  flow  is  illustrated  in  Figure  7.  The  figure  shows  the  63  transients 
comprising  a  data  set  on  the  left.  The  panel  on  the  right  shows  the  three  principal  polarizability 
rate  transients.  We  call  the  computer  program  that  accomplishes  this  transfonnation  MM/RMP. 
The  basis  of  MM/RMP  is  the  ubiquitous  point  dipole  model  approximation  for  approximating 
the  EMI  response  of  a  small  conducting  and  magnetically  penneable  object.  By-products  of  this 
modeling  process  are  estimates  of  the  target  position  and  attitude  angles.  The  dipole  model 
approximation  is  widely  used  within  the  UXO  community  for  interpreting  UXO  [6] -[10].  A 
particularly  thorough  and  systematic  treatment  of  the  theory  can  be  found  in  Chapter  2  of 
Leonard  Pasion’s  Ph.D.  thesis  [11].  The  algorithm  used  in  MM/RMP  was  developed  by  J. 
Torquil  Smith  at  LBNL  [12].  The  basic  theory  for  the  dipole  approximation  of  EMI  response  for 
small  objects  is  summarized  in  Figure  8.  We  outline  the  essential  elements  of  the  theory  below: 

1.  We  can  approximate  the  behavior  of  the  target  as  a  point  dipole  characterized 
with  a  time-  or  frequency-dependent  polarizability  tensor  (P). 

2.  The  primary  magnetic  field  (Hp)  “illuminates”  the  target  object,  which  is  assumed 
to  be  very  highly  conducting  and  (possibly)  permeable.  In  Figure  8,  the  primary 
field  is  generated  by  transmitter  loop  (red)  carrying  a  time  varying  current. 

3.  As  suggested  in  Figure  8,  the  polarizability  tensor  P  is  a  second  rank  symmetric 
tensor  having  six  independent  elements.  In  such  cases,  there  exists  a  coordinate 
system  called  the  “principal  axes”  coordinate  system  in  which  the  polarizability 
tensor  is  diagonal  with  only  three  elements.  The  “principal  axes”  coordinate 
system  and  the  observation  coordinate  system  are  related  by  means  of  an 
orthogonal  transfonnation  (T).  The  orthogonal  transformation  can  be  expressed 
in  terms  of  three  elementary  rotation  angles  (cp,0,i[/)  sometimes  called  the  “Euler 
Angles.”  At  any  particular  time  or  frequency,  the  response  at  the  receiver  is  a 
function  of  nine  parameters:  1)  the  three  coordinates  of  the  target  position,  2)  the 
three  Euler  Angles,  and  3)  the  three  diagonal  elements  of  the  polarizability  tensor 
evaluated  at  a  particular  time  or  frequency. 

4.  Given  sufficient  independent  measurements  of  the  secondary  field  B,  one  can 
estimate  the  nine  unknown  parameters  enumerated  in  6  parameters  using  well- 
known  principles  of  nonlinear  and/or  linear  inversion. 
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Principal  Polarizability  Transients 
(“Intrinsic”  target  parameters) 


MM/RMP 

(Pt  Dipole  Model) 


Tgt  Position/Attitude 
(“Extrinsic”  Parameters) 


Figure  7.  Parameter  extraction  using  MM/RMP. 

The  input  data  are  transients  (63)  acquired  during  a  single  static-mode  measurement  with  the 
MM.  The  outputs  are  the  principal  polarizability  rate  transients,  target  position,  and 

target  attitude. 


B(r-r„  ,r0-f;i)  =  ^G(r-r0)-{p(f)H'(f0-f)} 
r  =  Rx  position;  r'  =  Tx  position;  r0  =  target  position 
G(F-F0)  =  Dipole  Green's  function 
P  (f )  =  Polarizability  rate  tensor  (symmetric  w/  6  unknowns) 
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H?  (r0  -r)  =  Primary  magnetic  field  intensity  (function  ofr0). 
T  ( <f>, 3, if/)  =  Orthogonal  transformati on  (3  unknowns ) 
p  (t),p  (?),Pj(f)  =  Principal  polarizability  transients 


Figure  8.  Summary  of  the  theory  for  approximating  the  response  of  a  small  conducting 

and  permeable  object  with  a  point  dipole. 
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7.3.1  Meta-Parameters 


Scalar  Meta-Parameters— Many  have  found  success  with  target  discrimination  using  scalar 
parameters  derived  from  much  simpler  instruments.  Moments  of  the  principal  polarizability 
transients  are  easy  to  compute  numerically  and  very  robust  with  respect  to  noise.  We  have 
benefitted  from  the  insight  on  the  relationship  of  moments  to  meaningful  target  features  provided 
in  a  paper  by  Smith  and  Jones  [13].  Mathematically,  the  nth  moment  of  the  polarizability  is 
defined  as 


Pn  =  J  t"  P(t)dt  ;  where  P(t)  is  a  polarizability  rate  transient. 

We  calculate  the  0th  and  1th  moments  for  each  of  the  three  principal  polarizability  transients 
thereby  producing  six  scalar  “meta-parameters”  (Pox,  Poy,  P(t,  Pix,  Piv,  Pjz)  where  the  subscripts 
(. x ,  y,  and  z )  denote  the  principal  axes  coordinates. 

For  each  of  these  scalar  parameter  sets  (Pq,  Pi),  we  calculate  two  additional  scalar  values,  two  of 
which  relate  to  target  shape: 

P  ■ 

1.  Target  Aspect:  Put  =  0,1  — This  is  a  measure  of  the  aspect  or  elongation 

piT 

of  the  target.  Values  of  P,r>  1  suggest  rod-like  targets  while  P,r<  1  suggest  flat 
targets. 

(p.  —p.  ) 

2.  Target  Eccentricity:  PiE  =  LJ*rJ^;i  =  o,i  — This  is  a  measure  of  the 

pix 

eccentricity  of  the  target.  A  target  with  good  symmetry  with  respect  to  the 
principal  x  axis  will  have  P  0E«  1  • 

7.4  CLASSIFIER  AND  TRAINING 

Discrimination  starts  with  the  estimation  of  target-related  parameters  from  our  data.  When 
viewed  in  the  principal  coordinate  system,  the  polarizability  of  a  UXO  will  manifest  itself  by 
exhibiting  a  single  large  (major)  polarizability  transient  over  two  smaller  (minor)  transients. 
Clutter-like  objects  do  not  exhibit  such  symmetry  properties. 

With  good  data,  an  interpreter  can  easily  discriminate  between  items  that  are  most  likely  TOIs 
and  items  that  are  most  likely  clutter.  The  visual  (and  numerical)  clues  are: 

•  Object  Size— Size  as  suggested  by  the  three  scalar  values  (Pqx,  Poy,  Poz)  can  be 
used  to  narrow  the  possibilities  to  one  of  three  size  classes. 

•  Symmetry  (Eccentricity)— In  most  cases,  the  principal  polarizability  curves 
should  indicate  a  rod-like  body  having  a  large  major  polarizability  curve  and  two 
smaller  minor  polarizability  curves  of  roughly  the  same  size  and  shape.  This  is 
the  case  for  all  examples  we  have  shown  thus  far.  Note  that  the  symmetry  or 
eccentricity  property  is  a  function  of  time. 
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•  Aspect  Ratio— This  is  the  ratio  of  the  major  curve  to  the  geometric  mean  of  the 
minor  curves  and  is  a  function  of  time. 

•  Major  Polarizability  Shape— The  shape  of  the  largest  principal  polarizability 
curve  is  often  very  distinctive. 

7.4.1  Artificial  Neural  Net  (ANN)  Classifier 

We  used  an  ANN  to  generate  a  binary  decision  regarding  whether  a  particular  target  is  a  TOI  or 
not.  We  needed  to  choose  a  set  of  feature  parameters  from  the  data  to  feed  to  the  neural  network. 
For  this  purpose,  we  used  the  commercially  available  software  package,  Statistica  version 
6. 0/9.0.  We  analyzed  scalar  polarizability  parameters  extracted  from  static  MM  data  sets 
collected  over  2177  targets. 

Our  objective  was  to  combine  a  basic  statistical  analysis  of  the  data  with  expert  knowledge  about 
the  data  and  the  ANN  choice  of  parameters  to  select  a  subset  of  parameters  to  use  for  the  ANN 
analysis.  Input  parameters  should  be  selected  to  provide  the  best  separation  between  clutter  and 
TOI  signatures.  In  order  to  keep  the  number  of  connection  weights  in  the  ANN  small,  we  want 
to  identify  the  smallest  set  of  input  parameters  that  can  provide  effective  classification. 

We  visually  classified  2177  targets  as  UXO  or  clutter.  Next  we  looked  at  the  correlation  of 
parameters.  Ideally  we  want  parameters  that  are  not  correlated  with  each  other  as  input 
parameters  since  this  means  they  include  redundant  infonnation.  Next,  we  looked  at  the 
sensitivity  of  the  classification  to  the  input  parameters.  We  ran  several  trials  and  measured  in 
how  many  trials  particular  features  were  considered  important.  Our  final  feature  vector  includes 
nine  robust  parameters  that  capture  (in  scalar  form)  the  properties  of  target  size  (polarizability  at 
t=0  [P0]  -  3  axes),  target  aspect  (P0_R),  target  symmetry  (P0_E),  and  target  polarizability  time 
persistence  (P1/P0  -  3  axes): 

(POx,  POy,  POz,  POR,  POE,  PIE,  Plx/POx,  Ply/POy,  Plz/POz) 

After  the  results  of  the  training  dig  were  distributed  to  those  involved  in  the  classification 
processing,  we  had  just  sufficient  data  so  that  we  were  able  to  train  a  network  totally  on  data  for 
which  the  ground  truth  was  known. 

7.4.2  Target  Types  for  Library  Matching 

We  supplemented  the  ANN  processing  with  a  library  matching  algorithm  in  order  to  identify 
some  items  that  were  missed  by  the  neural  network.  Initial  reconnaissance  dig  infonnation,  later 
buttressed  by  results  from  the  training  dig,  identified  four  primary  TOI  types.  We  developed  our 
initial  “type”  polarizability  curve  library  from  the  static  measurements  we  acquired  with  the  MM 
in  the  test  pit  and  the  test  strip.  Using  the  specimen  targets  provided  at  SLO  we  measured  the 
free-air  response  of  each  target  placed  at  four  different  attitudes  and  two  different  depths.  In 
addition,  we  acquired  static  measurements  over  a  specimen  of  each  target  type  buried  in  the 
calibration  strip  several  times  during  the  course  of  our  demonstration. 
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Figure  9.  Principal  polarizability  curves  for  the  four  munitions  of  primary  interest  at  SLO. 

Each  curve  set  is  derived  from  an  ensemble  of  14  static  measurements  over  targets  at  different 
depths  and  orientations.  The  heavy  curves  (red/green/blue)  are  the  geometric  mean  curves 
computed  by  taking  the  geometric  mean  of  their  respective  ensemble. 


7.4.3  Classification  Results— Training  and  Test 

As  we  indicated  above,  we  ended  up  training  two  networks: 


1.  ANN  Network  1:  This  ANN  was  trained  with  test  pit  and  test  strip  data  (58  data 
points),  data  from  the  test  dig  (208  points),  and  12  data  points  that  were  visually 
classified  as  60  mm  no  fins  (NF).  Although  we  had  MM  static  points  associated 
with  only  154  of  the  targets  in  the  training  dig,  many  of  those  targets  had  two  or 
more  MM  data  points  associated  with  the  target.  By  using  these  (54)  repeated 
data  points,  we  “augmented”  our  training  set  so  that  it  had  278  targets. 

2.  ANN  Network  2:  This  ANN  was  trained  with  a  total  of  563  targets  comprised  of 
the  58  points  from  the  test  pit  and  test  strip,  the  208  points  (as  above)  from  the 
training  dig,  and  296  points  that  were  visually  classified. 


Figure  10  shows  the  receiver  operating  characteristics  (ROC)  curves  that  result  from  applying 
the  two  ANNs  to  their  respective  training  data.  Note  that  in  both  cases,  the  ANN  leaves  a  few 
“hard”  false  negatives  that  would  require  us  to  dig  a  high  percentage  of  all  the  targets  before 
digging  the  last  TOI.  However,  after  applying  the  library  matching,  the  hard  false  negatives  are 
moved  closer  to  the  knee  (operating  point)  of  the  curve,  thereby  dramatically  reducing  the 
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number  of  targets  we  need  to  dig  before  we  dig  the  last  TOI  (100%  detection  point).  However, 
the  major  benefit  of  using  library  matching  is  that  it  reduces  the  number  of  false  negatives. 
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Figure  10.  ROC  curves  showing  the  results  of  applying  ANN  and  the  ANN  +  library  curve 
matching  to  their  respective  training  sets  after  training. 

The  curve  library  contains  five  types  (60  mm,  60  mm  NF,  2.36-inch,  81  mm,  4.2-inch). 

7.5  DATA  PRODUCTS 

The  principal  data  product  arising  from  the  processing  is  a  prioritized  dig  list  that  can  be  scored 
against  ground-truth.  We  have  tried  to  emphasize  throughout  this  section  that  our  procedures 
and  ability  to  construct  a  prioritized  dig  list  have  evolved  during  the  course  of  the  three 
demonstrations  we  have  conducted  and  have  reported  on  here.  Indeed,  the  SLO  demonstration 
has  led  us  to  develop  and  document  each  step  in  the  classification  process. 

While  the  prioritized  dig  list  is  the  primary  product  of  the  demonstrations,  there  are  also  many 
other  data  products  required  in  order  to  adequately  document  and  measure  the  performance  of 
the  MM  against  the  various  performance  objectives  that  we  have  enumerated  in  Section  4. 
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8.0  PERFORMANCE  ASSESSMENT 


In  this  section  we  will  assess  the  performance  of  the  MM  system  against  the  perfonnance 
objectives  contained  in  Table  1. 

8.1  TARGET  DETECTION 

At  SLO,  there  were  187  seeded  targets.  The  MM  failed  to  detect  four  of  those  seed  targets.  The 
four  targets  we  missed  were  all  60  mm  mortars.  Therefore,  we  achieved  the  stated  performance 
goal  of  Pd  >=  98%. 

It  is  instructive  to  look  at  the  reasons  we  failed  to  detect  these  four  targets  (SLO  targets  59,  478, 
410,  and  16).  All  four  were  60  mm  mortar  bodies.  Target  410  is  typical  of  the  other  three.  The 
issue  is  summarized  graphically  in  Figure  11.  The  anomaly  is  obvious  in  both  the  map  and  the 
profiles.  But  it  fails  to  rise  above  the  experimentally  verified  detection  threshold.  The  “expert” 
interpreter  would  have  recognized  this  anomaly  as  likely  arising  from  a  deep  target  because  of  its 
broad  extent  and  coherence,  notwithstanding  its  low  amplitude.  A  look  at  the  recovered  target 
object  (Figure  12)  explains  why  this  target  had  a  lower  detection  signal  amplitude  than  we 
expected  for  a  60  mm  object.  The  seeded  target  was  a  60  mm  mortar  (body  only)  target.  We 
defined  the  detection  threshold  based  on  the  signal  amplitudes  for  a  60  mm  M49A4,  the  type  of 
60  mm  mortar  seeded  at  APG  and  YPG,  buried  in  the  test  lanes  at  SLO,  and  provided  at  SLO  for 
free-air  tests.  The  mortar  body  target  has  an  electromagnetic  size  that  is  approximately  half  the 
size  of  the  complete  mortar  including  fins  and  a  fuse.  Obviously,  the  detection  threshold  was  set 
too  high.  However,  had  we  set  the  threshold  to  1  uV/Am2  we  would  have  easily  doubled  or 
perhaps  tripled  the  number  of  targets  detected. 


Figure  11.  Figure  showing  one  target  that  went  undetected  in  the 

MM  dynamic  survey. 

Target  410  is  readily  discernible  on  both  the  map  and  the  profiles.  Although  it  has  a  broad  areal 
extent  indicating  a  deep  target,  its  amplitude  fails  to  rise  above  the  detection  threshold. 
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Figure  12.  Photograph  of  the  recovered  60  mm  “body” 
associated  with  SLO  target  number  410. 

8.2  ANALYSIS  AND  CLASSIFICATION  PERFORMANCE 

The  SLO  project  permitted  us  to  develop  a  methodology  that  achieves  the  performance 
objectives  set  out  in  the  ESTCP  demonstration  plan.  We  base  the  following  discussion  on  the 
scoring  results  that  we  received  from  IDA  in  early  November  2009.  In  Figure  13,  we  have 
reproduced  and  annotated  the  IDA  ROC  curves.  These  ROC  curves  summarize  the  scoring 
results  representing  a  partial  dig  (1064  items  from  a  list  of  1408)  of  the  MM  target  list. 


Geometries  MetalMapper  ANN-LM-A  Test  Set  AIMuntions  Geometnc*  MetalMapper  ANN-TKLM  TestSet  AIMuntoon* 


A)  SLO  Discrimination  Results  -  Target  List  1 _ B)  SLO  Discrimination  Results  -  Target  List  2 

Figure  13.  IDA  ROC  curves  for  the  two  discrimination  stage  target  lists  submitted  in 
connection  with  the  SLO  classification  study. 

The  differences  in  the  lists  are  the  result  of  using  different  data  sets  for  training  the  ANN  and 
different  versions  of  the  ANN  software.  In  the  insets,  we  have  identified  the  four  false  negative 
targets  that  occur  beyond  the  MM  threshold.  Note  that  the  same  four  targets  are  involved  in  both 

target  lists  (albeit  in  different  order). 


4  The  test  data  used  by  IDA  consists  of  1064  targets  of  which  206  were  designated  TOI  and  the  balance  (858)  as 
clutter  or  non-ordnance  items. 
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8.2.1  Maximize  Correct  Classifications— Munitions  (see  Section  4.2) 

The  ROC  curves  (Figure  13)  show  that  at  our  operating  threshold  we  had  four  false  negatives, 
two  of  which  were  very  “hard”  in  the  sense  that  our  classifier  indicated  that  it  was  highly 
probable  that  these  objects  were  clutter. 


We  have  examined  the  dig  results  from  the  four  targets  associated  with  the  hard  negatives.  In  the 
expanded  ROC  curves  (inset  Figure  13,  A  and  B)  the  four  targets  are  identified  according  to  their 
MM  ID  and  Master  List  IDs,  respectively,  with  the  “/”  character  as  a  separator. 


8.2. 1.1  MM-ID  292/Master  ID  1502  (37mm) 


The  37  mm  target  is  the  only  target  that  is  an  obvious  “miss”  that  could  have  (should  have)  been 
avoided  by  a  comprehensive  visual  examination  of  the  polarizability  curves.  In  both  target  lists, 
the  37  mm  target  (the  only  one  in  the  test  data  set)  was  very  strongly  identified  as  clutter  (non¬ 
munition).  The  problem  is  that  our  classifier  was  not  trained  to  identify  a  target  of  the  size  and 
shape  of  the  37  mm.  In  a  retrospective  study,  we  added  the  polarizability  curves  for  a  37  mm 
from  our  library  of  munitions  types  for  APG  (see  Figure  14).  Our  library  matching  algorithm 
identified  the  37  mm  with  very  high  probability.  Therefore,  we  conclude  that  we  will  be  able  to 
detect  and  classify  37  mm  targets  using  library  matching  provided  we  add  a  37  mm  type  to  our 
matching  library  (a  sixth  type)  and  that  we  include  specimens  of  the  nine-component  feature 
vector  extracted  from  static  measurements  of  37  mm  projectiles. 


J7nim(DF) 


SLOStatA00292-Principal  Polarizability  Curves  (Type  1) 


dPI/dt  -  SMB1  r 

Errorl 

dP2/dt 

SMB2 

Enor2^  dP3/dt 

SMB3 

T  Errof3 

A)  37mm  “Type”  Polarization  Curves  -  APG  B)  37mm  Polarization  Curves  -  MM  Cued  ID  292 

Figure  14.  Principle  polarizability  curves  for  a  37  mm  projectile. 

The  curves  on  the  left  (A)  represent  the  average  six  or  eight  curve  sets  extracted  from  data  sets 
observed  over  37  mm  projectiles  in  the  Calibration  Lanes  at  APG.  The  curves  on  the  right  were 
extracted  from  static  data  set,  SLOStatA00292.tem,  the  only  data  set  observed  over  this  37  mm 

target.  This  target  was  missclassified  as  clutter. 


29 


1?  Igl 

n 

5>  • 

J  rr 

A  ^ 

CQ~^.  L/'aJ  - 

I/i  .JT.  (PL 

ffl 

_ A, 

A. 

V* 

QX 

A)  Excerpt:  Field  notes  (p.  SL061  6/13/09) 


102 

101 


10° 


10'1 

10"2 


102  103  1  04 

I  (US) 

C) 

Figure  15.  Field  note  describing  target  near  1715  as  a  “rocket  shaft”  (A)  together  with  the 
polarizability  curves  for  1715  and  a  repeat  (2204). 

The  polarizability  curves  both  look  like  rocket  parts.  These  two  data  sets  are  within  a  meter  or 

twoofMM-ID  1718. 

8.2. 1.2  MM  ID  1177/Master  ID  775  (60  mm  mortar) 

The  second  hard  false  negative  is  the  60  mm  mortar  found  in  association  with  MM-ID  1177 
(Master  List  ID  775).  The  polarizability  curves  together  with  the  dig  photo  are  in  Figure  16. 

SLOStatAOl  177-Principal  Polarizability  Curves  (Type  1) 

102 
101 


I  10"1 


t  (US) 

Figure  16.  MM  cued  ID  No.  1177  and  the  corresponding  dig  photo. 

The  target  in  the  hole  is  a  60  mm  mortar.  The  polarizability  curves  appear  to  represent  the 

associated  tail  booms  and/or  fuse. 


SLOStatA02204-Pri ncipal  Polarizability  Curves  (Type  1) 
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SLOStatAOl  715-Principal  Polarizability  Curves  (Type  1) 
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SLOStatAOl  71 8-Principal  Polarizability  Curves  (Type  1) 


t(us) 


SLOStatAOl  782-Principal  Polarizability  Curves  (Type  1) 
r-  dPl/dt  SMBl  i  Efron  dP2/dt  SMB2  I 


Figure  17.  Polarizability  curves  and  dig  results  for  MM-ID  Numbers  1718  and  1782,  two  of 


four  false  negatives  in  both  target  lists  submitted  to  IDA. 


The  polarizability  curves  suggest  a  shallow  equi-dimensional  object  that  is  consistent  with  the  60 
mm  tail  boom  segments  sitting  on  the  board.  The  anomaly  amplitude  is  less  10  a  microtesla 
(p.T)/s,  which  means  it  is  only  slightly  above  the  detection  threshold  for  a  deep  mortar.  Yet  the 
solution  for  the  target  position  indicates  a  target  at  a  depth  of  13  cm.  It  appears  therefore  that  the 
anomaly  produced  by  the  tail  boom(s),  which  no  doubt  were  found  above  the  deep  mortar  shown 
in  the  photo,  totally  overpowers  the  low  signal  level  from  the  mortar.  This  is  a  case  where  the 
interference  from  several  shallow  sources  obscures  the  deeper  TOI.  The  best  and  only  chance 
we  have  of  mitigating  this  type  of  anomaly  is  to  recognize  that  it  is  a  complex  response  that 
cannot  be  analyzed  and  therefore  must  be  dug. 

8.2. 1.3  MM-ID  1718/Master  ID  1475  12.36-inch  rocket  motor) 

The  polarizability  curves  for  MM  targets  1717  and  1718  and  the  size  estimates  indicate  that  the 
parameters  most  likely  belong  to  the  60  mm  tail  boom  shown  in  the  dig  photo  rather  than  the 
rocket  motor.  The  polarizability  response  of  an  isolated  rocket  motor  is  significantly  larger  and 
shows  good  symmetry  and  large  aspect.  This  target  intrigued  us  partly  because  it  is  a  hard  false 
negative  but  also  because  the  dig  reports  that  the  target  was  found  laying  on  the  surface.  How 
then  could  we  have  missed  a  target  like  that?  In  looking  through  our  field  notes,  we  came  across 
a  note  for  file  SLOStatA01715,  a  static  data  point  acquired  in  the  immediate  vicinity  of  1718. 
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The  note  (see  Figure  15  A)  indicates  that  a  measurement  on  an  unpicked  point  was  acquired  near 
or  over  a  “rocket  shaft”  that  had  obviously  been  moved  recently  because  it  was  lying  on  top  of 
the  grass.  It  is  likely  that  a  rocket  motor  was  found  in  the  immediate  vicinity  (i.e.,  within  a 
radius  of  1  m  or  so)  and  lumped  in  with  the  60  mm  tail  boom.  The  ground  truth  indicates  the 
target  was  found  within  a  distance  of  0.3  m  of  the  location  of  MM  1718.  Our  theory  is  that  this 
object  was  moved  after  the  target  detection  survey,  so  this  does  not  represent  a  failure  of  our 
analysis  to  correctly  classify  the  object. 

8.2. 1.4  MM-ID  1782/Master  ID  1541  (2.36-inch  rocket  motor) 

Relatively  speaking,  target  number  MM-ID  1782  (Master  ID  1541)  is  the  least  important  of  the 
four  false  negative  targets  that  are  labeled  on  both  panels  of  Figure  13.  The  position  of  this 
target  is  roughly  the  same  on  both  ROC  curves  (Figure  13  A  &  B)  and  requires  only  a  few 
unnecessary  digs  past  the  operating  point  (dark  blue  dot)  to  get  to  the  target.  We  mention  it  here 
only  for  completeness.  We  would  like  to  note,  however,  that  the  target  (Master  ID  1541)  is 
similar  in  both  appearance  and  in  the  behavior  of  its  principal  polarizability  curves  to  target  SLO 
Master  ID  1322  that  was  belatedly  labeled  “MD”  after  a  more  careful  examination.  The 
polarizability  curves  and  photos  of  MM  targets  954  and  1782  provide  a  comparison  between  the 
two  targets,  one  labeled  “MD”  and  the  other  labeled  “TOI.”  The  similar  behavior  of  the 
polarizability  curves  suggests  that  target  1782  is  cracked  or  otherwise  physically  damaged  in  a 
way  similar  to  that  of  954.  Because  this  item  was  so  heavily  damaged,  it  no  longer  exhibits  the 
properties  we  expect  of  such  a  TOI.  In  fact,  this  item  is  so  heavily  damaged,  it  barely  classifies 
as  a  TOI  at  all. 


SLOStatA00954-Principal  Polarizability  Curves  (Type  1) 


SLOStatAOl 782-Principal  Polarizability  Curves  (Type  1) 
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Figure  18.  Polarizability  curves  for  two  2.36-inch  “empty”  warheads. 

The  target  on  the  left  is  from  the  training  dig;  the  one  on  the  right  is  from  the  test  dig.  Although 
target  1782/1541  was  deeper  than  954/1260,  both  targets  exhibit  similar  behavior  in  the  early 

time. 
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8.3  MAXIMIZE  CORRECT  CLASSIFICATION  OF  CLUTTER 


In  the  test  data  set,  there  were  a  total  of  857  clutter  objects.  The  results  from  ANN  2  (Figures 
10B  and  13B),  show  that  after  -300  unnecessary  digs,  we  have  excavated  all  the  TOIs  (Pd  = 
100%).  Therefore,  we  found  all  the  TOIs  after  a  total  of  506  digs  and  would  be  able  to  safely 
leave  the  remaining  557  clutter  targets  in  the  ground.  This  represents  a  reduction  of  65%  in  false 
alanns  and  thus  we  exceed  the  perfonnance  goal  of  a  30%  reduction. 

8.4  MINIMIZATION  OF  “CANNOT  ANALYZE”  POINTS 

Of  the  1063  targets  that  were  scored,  there  were  15  category  4  targets  (“cannot  analyze”).  Those 
were  targets  for  which  the  fit  statistic  was  less  than  75.  The  points  represent  1.4%  of  all  the 
targets  analyzed.  Using  the  ground  truth  from  the  test  dig,  we  determined  that  the  discrimination 
results  for  all  15  category  data  points  were  correct.  Although  we  do  not  feel  comfortable  with 
completely  eliminating  a  screening  rule  that  relegates  data  points  with  poor  fit  to  category  4,  in 
retrospect  we  think  that  we  were  too  conservative  in  assigning  the  Fit<75%  threshold.  By 
assigning  a  slightly  lower  threshold  (e.g.,  Fit  <  70%),  we  can  eliminate  half  the  category  4  points 
with  little  or  no  danger  of  introducing  a  false  negative.  In  any  case,  MM  has  clearly  met  the 
perfonnance  goal  of  >90%  of  all  targets  being  analyzable. 

8.5  DYNAMIC-MODE  SURVEY  PRODUCTIVITY 

Our  performance  objective  for  dynamic-mode  survey  productivity  was  1.2  acres/day.  In  Table  2, 
we  have  tabulated  the  work  time  for  each  of  the  sub-areas  that  we  defined  within  the  overall 
demonstration  area.  These  time  intervals  (At)  represent  the  time  between  the  first  and  last 
dynamic  calibration  of  the  day  (or  last  calibration  for  a  particular  sub-area  survey)  and  therefore 
include  time  for  breaks  and  breakdowns.  What  those  time  intervals  do  not  include  is  the  time  for 
set-up  and  take-down  each  day.  We  estimate  that  time  to  be  0.5  hr/day.  We  based  our 
production  estimates  on  the  number  of  work  hours.  At  SLO,  we  generally  worked  about  9-10  hr 
in  the  field.  Therefore,  we  converted  hourly  production  to  daily  production  by  dividing  by  9 
hr/day.  Based  on  the  table,  we  met  or  slightly  exceeded  our  production  goal  (0.054  ha/hr  =  0.49 
ha/day=  1.22  acre/day). 

The  table  shows  that  we  have  met  or  exceeded  our  performance  goal  for  dynamic-mode  survey 
productivity  (1.2  acre/day). 

Table  2.  Dynamic  survey  productivity  at  SLO. 


Sub-Area 

Date 

Wor 

i  Time 

Production 

File  No 

Blks 

Area 

Start 

End 

At 

At 

(hr) 

Rte 

(ha/hr) 

Rte 

(ha/d) 

1st 

Last 

(ha) 

NWA 

5/25/09 

12:33 

17:45 

5:12 

5.20 

0.035 

0.18 

1 

47 

2 

NWB 

5/26/09 

7:12 

14:51 

7:39 

7.65 

0.035 

0.27 

48 

101 

3 

NWC 

5/26/09 

15:48 

17:48 

2:00 

2.00 

0.046 

0.43 

102 

116 

6 

0.54 

NWC 

5/27/09 

7:16 

16:14 

8:58 

8.97 

117 

232 

NWC 

5/28/09 

6:59 

7:46 

0:47 

0.78 

233 

254 

SWA 

5/28/09 

9:13 

15:41 

6:28 

6.47 

0.042 

0.27 

255 

338 

3 

SA 

5/29/09 

6:59 

16:51 

9:52 

9.87 

0.055 

0.54 

339 

424 

6 
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Table  2.  Dynamic  survey  productivity  at  SLO  (continued). 


Sub-Area 

Date 

Wor 

i  Time 

Production 

File  No 

Blks 

Area 

Start 

End 

At 

At 

(hr) 

Rte 

(ha/hr) 

Rte 

(ha/d) 

1st 

Last 

(ha) 

SE 

5/30/09 

7:11 

12:14 

5:03 

5.05 

0.088 

0.54 

425 

516 

9.5 

0.855 

SE-A 

5/30/09 

12:52 

17:32 

4:40 

4.67 

517 

576 

E 

5/31/09 

6:01 

15:59 

9:58 

9.97 

0.068 

0.68 

577 

687 

7.5 

0.675 

NE 

6/1/09 

7:24 

17:13 

9:49 

9.82 

0.057 

0.56 

688 

785 

6.2 

0.558 

NE-A 

6/2/09 

7:41 

12:22 

4:41 

4.68 

0.035 

0.16 

786 

861 

1.8 

0.162 

N 

6/9/09 

5:42 

15:47 

10:05 

10.08 

0.071 

0.72 

862 

1061 

8 

NWB-Rep 

6/15/09 

8:45 

11:23 

2:38 

2.63 

0.103 

0.27 

1062 

1105 

3 

Totals 

87.83 

0.46 

(avg) 

56 

5.04 

(avg) 

8.6  STATIC-MODE  SURVEY  PRODUCTIVITY 

Our  performance  goal  at  SLO  was  to  acquire  greater  than  200  static  data  points  per  day.  Table  3 
shows  that,  at  SLO,  our  static  productivity  was  39  pts/hr.  At  SLO,  we  measured  494  pts  on  our 
best  day.5  The  improved  production  rates  reflect  the  front-mounted  antenna  deployment. 
Secondly,  we  decreased  the  stacking  time  by  a  factor  or  2  because  of  the  generally  better  signal- 
to-noise  ratio  (SNR)  at  SLO. 

Table  3.  Static-mode  survey  productivity  at  Camp  SLO. 


Area 

Date 

Work  Time 

Production 

File  Index 

Start 

End 

At 

At 

(hr) 

Rte 

(pts/hr) 

Rte 

(pts/d) 

1st 

Last 

Tst  Strip 

6/3/09 

7:27 

8:40 

1:13 

1.22 

17 

201 

2 

22 

Tst  Pit 

6/3/09 

9:03 

10:18 

1:15 

1.25 

27 

23 

56 

Tst  Strip 

6/8/09 

7:39 

8:35 

0:56 

0.93 

13 

242 

57 

68 

NWC 

6/8/09 

9:22 

14:25 

5:03 

5.05 

43 

1 

217 

NWC 

6/10/09 

14:37 

17:40 

3:03 

3.05 

41 

197 

364 

489 

NWB 

6/10/09 

8:23 

11:48 

3:25 

3.42 

43 

218 

363 

NWA 

6/10/09 

17:41 

18:34 

0:53 

0.88 

40 

490 

524 

NEPlus 

6/11/09 

8:35 

15:21 

6:46 

6.77 

40 

405 

525 

795 

E 

6/11/09 

15:33 

18:18 

2:45 

2.75 

57 

796 

952 

E 

6/12/09 

8:13 

13:10 

4:57 

4.95 

51 

494 

953 

1205 

SEPlus 

6/12/09 

14:18 

18:08 

3:50 

3.83 

60 

1206 

1434 

Splus 

6/13/09 

8:30 

17:01 

8:31 

8.52 

50 

449 

1435 

1859 

N 

6/14/09 

9:46 

17:25 

7:39 

7.65 

42 

375 

1860 

2178 

Stat-Rep 

6/15/09 

12:40 

18:42 

6:02 

6.03 

17 

155 

2179 

2282 

Stat-Rep 

6/16/09 

8:47 

17:33 

8:46 

8.77 

24 

216 

2283 

2492 

Totals 

65.07 

39 

(avg) 

354 

(avg) 

2559 

(avg) 

5  At  SLO,  our  access  to  the  site  was  unrestricted.  We  generally  stayed  a  full  10  hr  at  the  site.  Daily  production  rates 
at  SLO  partly  reflect  longer  work  days  than  at  APG.  For  comparing  production  rates  it  is  better  to  use  the  hourly 
rates. 
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9.0 


COST  ASSESSMENT 


9.1  COST  MODEL 

In  Table  4  we  provide  a  cost  model  for  the  deployment  of  the  MM  technology. 

Table  4.  Cost  model  for  an  MM  deployment. 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Instrument  cost 

Component  costs  and  integration  costs 

•  Engineering  estimates  based  on  current  development 

•  Lifetime  estimate 

Track  consumables  and  repairs 

$100K-$120K 

Mobilization  and 
demobilization 

Cost  to  mobilize  to  site 

Derived  from  demonstration  costs 

$20K/$15K 

Instrument  setup  costs 

Unit:  $  cost  to  set  up  and  calibrate 

Data  Requirements: 

•  0.5  hr 

•  2  people 

•  twice  per  day 

$500/day 

Survey  costs 
(Dynamic) 

Unit:  $  cost  per  hectare 

Data  requirements: 

•  Hours  per  hectare  -  18  hr 

•  Personnel  required  -  2 

$7.2K/ha 

Survey  costs 
(Static) 

Unit:  $  cost  per  300  targets 

Data  requirements: 

•  Hours-  9  hr 

•  Personnel  required  -  2 

$3.6K 

Detection  data 
processing  costs 

Unit:  $  cost  per  hectare 

Data  requirements: 

•  Time  required  -  18  hr 

•  Personnel  required  -  1 

$6.6K 

Discrimination  data 
processing  costs 

Unit:  $  per  300  anomaly  anomalies 

•  Time  required  -  9  hr 

•  Personnel  required  -  1  unit 

$3.3K 
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9.1.1  Equipment  Costs 


Major  subsystems  comprising  the  MM  system  together  with  their  respective  cost  estimates  are 
listed  in  Table  5. 


Table  5.  Major  components  comprising  a  MM  system. 


MM  Component  Name 

Cost 

Antenna  platform  and  DAQ  system 

$65K 

RTK  GPS  system 

$30K 

Platform  attitude  sensor 

$5K 

Vehicle  deployment 

$5K-$20K 

Software 

$10K-$20K 

Antenna  Platform  and  DAQ— The  antenna  platfonn  and  the  DAQ  comprise  the  proprietary 
hardware.  The  platform  can  be  deployed  as-is  as  a  human-powered  cart.  The  instrument 
containing  the  DAQ  can  be  mounted  on  a  pack-frame  and  carried  by  an  operator. 

RTK  GPS  System— The  location  of  the  antenna  platfonn  is  nonnally  measured  using  an  RTK 
GPS  system  in  which  the  roving  antenna  is  fixed  to  the  antenna  platform. 

Platform  Attitude  Sensor— We  use  an  attitude  sensing  module  that  contains  a  two-axis 
accelerometer  to  measure  pitch  and  roll  angles  and  a  three-axis  fluxgate  magnetometer  that 
provides  reference  heading  to  magnetic  north. 

Deployment— The  MM  can  be  deployed  either  as  a  man-powered  cart  or  as  a  vehicle-towed 
system.  There  is  a  large  range  of  possibilities  for  tow  vehicles  and  therefore  we  have  provided  a 
cost  range  for  the  deployment. 

Software— Map  data  is  processed  using  Geosoft  OM  software.  We  supply  a  set  of  GXs  that 
pennit  MM  data  files  to  be  loaded  into  a  GDB.  Target  parameters  are  extracted  using 
Geometries’  proprietary  modeling  software  based  on  the  point  dipole  model.  Results  of  this 
target  parameterization  are  placed  in  a  Microsoft  Access  database. 

9.1.2  Mobilization  and  Demobilization 

The  cost  of  mobilization  includes  the  cost  of  equipment  preparation  and  checkout  plus  the  cost  of 
transportation  to  the  job  site.  The  estimated  cost  includes  labor,  transportation,  and  per  diem. 
Demobilization  does  not  include  cost  for  equipment  preparation. 

9.1.3  Instrument  Setup  and  Calibration 

Instrument  setup  and  calibration  procedures  require  approximately  one-half  hr  at  the  beginning 
and  at  the  end  of  each  field  day.  These  procedures  include  setting  up  the  GPS  system, 
conducting  operation  checks,  and  perfonning  static  and  dynamic  calibration  surveys. 
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9.1.4  Dynamic  Survey  Costs 


The  cost  estimate  is  based  on  a  crew  cost  of  $3600/day.  The  estimated  areal  production  is  based 
on  survey  profiles  on  0.75  m  offsets.  Survey  speeds  are  less  than  0.5  m/s.  Production  rates  can 
be  improved  by  increasing  the  survey  speed.  Production  of  up  to  1  ha/day  is  possible  at  a  survey 
speed  of  1  m/s. 

9.1.5  Static  Survey  Costs 

The  cost  of  static  surveys  is  based  on  an  average  production  rate  of  300  static  pts/day.  At  SLO 
we  averaged  well  over  300  pts/day  with  the  antenna  array  mounted  to  the  front  loader  of  a 
Kabota  four-wheel  drive  tractor.  Production  rates  will  vary  depending  on  the  type  of 
deployment. 

9.1.6  Detection  Data  Processing 

A  single  data  analyst  can  process  the  data  produced  by  an  MM  crew.  It  requires  approximately 
one  hour  of  the  analyst’s  time  per  hour  of  field  data  production.  Processing  includes  loading  the 
data,  producing  a  detection  map,  and  picking  targets.  It  also  includes  sequencing  the  target  lists 
for  static  surveys. 

9.1.7  Discrimination  Data  Processing 

As  with  detection  surveys,  a  competent  data  analyst  is  able  to  keep  up  with  the  processing  of  the 
data  acquired  by  the  MM  field  crew  (-300  static  pts/day).  Costs  are  figured  accordingly.  Note 
that  these  costs  do  not  include  the  costs  of  training  the  discrimination  algorithm. 

9.2  COST  DRIVERS 

The  cost  estimates  in  Table  4  use  higher  labor  costs  that  reflect  the  personnel  involved  in  early 
demonstrations.  These  labor  costs  will  no  doubt  drop  by  as  much  as  one-third  as  field  operations 
and  data  processing  tasks  become  routine  and  are  accomplished  by  lower-level  personnel.  The 
productivity  of  detection  surveys  is  likely  to  improve  by  as  much  as  a  factor  of  2  as  we  increase 
survey  speed.  And  finally,  the  productivity  of  the  data  analyst  stands  to  improve  markedly  as 
processing  tasks  are  integrated  and  automated.  It  is  likely  that  cost  of  processing  will  decrease 
by  as  much  as  50%. 

9.3  COST  BENEFIT 

The  cost  benefit  of  employing  advanced  EMI  technology  such  as  the  MM  can  be  realized  only  to 
the  extent  that  regulators  gain  sufficient  confidence  in  the  technology  that  they  are  comfortable 
letting  targets  that  have  been  classified  “High  Confidence  Not  Munitions”  remain  undug  in  the 
ground.  Based  on  the  cost  estimates  provided  in  Table  4,  the  unit  cost  for  cued  ID  analysis  of  a 
target  with  the  MM  is  approximately  $20/target  with  the  costs  likely  to  drop  rapidly  down  to 
$  10/target.  The  unit  cost  to  dig  has  been  estimated  to  run  between  $100-$200/target.  The 
scoring  results  for  the  MM  showed  that  fully  98%  of  the  munitions  would  have  been  dug  with  a 
corresponding  false  positive  dig  rate  of  less  than  10%  if  the  targets  had  been  dug  according  to  the 
prioritized  dig  list.  This  suggests  that  by  employing  such  a  dig  list  in  a  very  conservative  manner 
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at  least  half  of  the  targets  could  be  left  in  the  ground,  thereby  reducing  the  digging  costs  by  half 
Since  the  MM  static  survey  and  data  analysis  costs  are  approximately  10-20%  of  the  cost  to  dig, 
the  overall  cost  of  remediation  would  be  reduced  by  30-40%  (i.e.,  dig  50%  of  targets  and 
conduct  a  cued  ID  target  survey  with  the  MM  of  100%  of  the  targets  at  a  cost  of  10-20%  of  the 
cost  to  dig). 
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10.0  IMPLEMENTATION  ISSUES 


In  Section  9,  we  highlighted  the  most  important  issue  affecting  the  implementation  of  the  MM 
technology.  Clearly,  the  need  is  to  prove  to  the  authorities  that  regulate  and  supervise  the 
remediation  of  sites  contaminated  with  UXO  that  the  MM  technology  can  significantly  reduce 
the  cost  remediation.  We  think  that  the  ESTCP  PO  together  with  regulators  and  demonstrators 
are  making  some  headway  with  their  live  site  demonstration  program.  Thus  far,  successful 
demonstrations  at  Camp  Siebert,  AL;  Camp  SLO,  CA;  and  Camp  Butner,  NC,  have  been 
completed.  The  ESTCP  PO  plans  to  continue  the  live  site  demonstration  program,  and  the  MM 
will  be  used  extensively. 

Even  with  the  reduction  in  the  unit  survey  costs,  the  cost  for  deploying  the  MM  will  remain 
significantly  higher  than  the  cost  for  commercially  available  EMI  technology  such  as  the  EM6 1 . 
Clearly,  this  higher  cost  is  justified  provided  that  regulating  authorities  accept  the  discrimination 
capability  of  the  system  and  use  the  results  to  reduce  the  number  of  unnecessary  digs.  UXO 
contractors  will  no  doubt  begin  to  deploy  advanced  EMI  systems  such  as  the  MM  only  when 
regulating  authorities  start  specifying  the  use  of  such  technology  in  the  big  requests  relating  to 
UXO  site  remediation.  Clearly,  it  is  important  that  the  MM  participated  in  the  demonstration  at 
SLO  and  performed  well.  It  is  equally  important  that  several  other  data  processing 
demonstrators,  using  MM  data,  performed  equally  as  well. 

The  MM  system  is  available  as  a  commercial  product  by  Geometries,  Inc.  It  can  be  deployed  in 
the  field  using  personnel  with  the  same  technical  skills  as  those  who  routinely  conduct  UXO- 
related  geophysical  mapping  surveys.  In  some  respects,  data  processing  is  more  demanding. 
However,  we  are  confident  that  this  too  can  be  routinely  perfonned  by  geophysical  data 
processors  with  a  few  days  training. 
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APPENDIX  A 
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Fax:  (408)  954-0902 

E-mail:  markp@geometrics.com 

Principal  Investigator 

Skip  Snyder 

Snyder  Geoscience 

67 1  Crescent  Court 

Grand  Junction,  CO  81505 

Phone:  (970)  254-0330 

E-mail:  skips@bresnan.net 

Co-Principal  Investigator 

David  George 

G&G  Sciences 

873  23rd  Road 

Grand  Junction,  CO  81505 

Phone:  (970)  263-9714 

E-mail:  dgeorge@ggsciences.com 

Co-Principal  Investigator 

ESTCP 

ESTCP  Program  Office 

901  N.  Stuart  Street,  Suite  303 
Arlington,  VA  22203 

Phone:  (703)  696-2117 

Fax:  (703)  696-2114 

Program  Office 

A-l 


ESTCP  Office 

901  North  Stuart  Street 
Suite  303 

Arlington,  Virginia  22203 

(703)  696-2117  (Phone) 
(703)  696-2114  (Fax) 

E-mail:  estcp@estcp.org 
www.serdp-estcp.org 


